1. Industrial Field of Utilization
The present invention relates to a picture synthesizing apparatus and method.
2. Related Art of the Invention
To present a picture with a presence so that the viewer feels as if seeing the actual scene, it is important to present the image corresponding to the changes of viewing point so that the surrounding scene may be looked around when the viewer turns the head.
When a picture is synthesized from a three-dimensional model of an object or a reflection model of a surface hitherto used in computer graphics or the like, it is possible to synthesize the picture from an arbitrary viewpoint.
It is, however, often impossible to synthesize a picture of high quality in real time, depending on the instruction of move of viewing point of the viewer, due to limitation of the performance of the apparatus and the like.
Another problem is that the generated picture by computer graphics CG is an artificial and unnatural picture quality as compared with the natural picture taken by a camera or the like.
To solve these problems, it has been attempted to take in and utilize a CG picture preliminarily prepared by taking a processing time or a natural picture taken by a camera, in a synthesized picture.
Conventionally, such known arts included image synthesizing apparatuses for cutting out a picture depending on the direction of viewing point and angle of field of view indicated by the viewer, from a wide-angle plane picture taken by a wide-angle lens, or a columnar projected picture taken by rotating 360 degrees in the horizontal direction around one viewing point known as a panoramic picture.
These arts are described below. FIG. 11 is a diagram showing a method of cutting out a synthesized picture from a wide-angle plane picture. In the diagram, reference numeral 1101 denotes a plane picture taken by a wide-angle lens or the like, and the X-axis is defined in the horizontal direction with .smallcircle. at the center, and the Y-axis, in the vertical direction. Suppose to cut out a picture when rotated by .theta. degrees in the horizontal direction from the viewing point 1102. In this case, the center of a cut-out region 1103 is supposed to be P, and the right end and left end of the X-axis to be R and L, respectively. The X-coordinates of points R and L, Rx and Lx, are determined in the following formulas, supposing the angle of field of view in the horizontal direction to be .varies. degrees, and the distance from viewing point 1102 to the center .smallcircle. of the plane picture, that is, the focal length in taking picture to be F. EQU Rx=F.multidot.tan(.theta.-.varies./2) EQU Lx=F.multidot.tan(.theta.+.varies./2) [1]
where the angle of rotation is positive in the left-thread direction about the Y-axis.
Similarly, the Y-coordinates of the upper end and lower end of the cut-out coordinates in the vertical direction, Uy, By, are calculated in the following formula, supposing the vertical field angle to be .beta.. EQU Ux=F.multidot.tan(-.beta./2) EQU Bx=F.multidot.tan(.beta./2) [2]
Hence, by cutting out the region designated by Rx, Lx, Uy, By, depending on the angle of rotation .theta., a picture corresponding to the horizontal rotation of the viewing point can be synthesized.
Next is described a method of obtaining a synthesized picture in a certain viewing line direction from a panoramic picture. FIG. 12a is a schematic diagram showing the relation between a panoramic picture seen from above and a synthesized picture. Reference numeral 1201 is a panoramic picture projecting an image of 360 degrees in the horizontal direction on a circular cylinder. Various methods are known to create a panoramic picture, including a method of taking while rotating the photo detector of a vertical slit by 360 degrees, and a montage method of splicing plane pictures taken by rotating the camera or the like around the receiving plane by specific angles by deforming geometrically. The panoramic picture 1201 is a two-dimensional picture with angle of rotation of .theta. about the viewing point and height of h in the vertical direction. FIG. 12b is a plane development of this panoramic picture.
In FIG. 12a, the picture of observing the .theta.0 direction from the viewing point 1202 is a picture 1203 projecting the panoramic picture on a tangential plane of a circular cylinder.
Supposing the horizontal and vertical axes of a synthesized picture to be U, V and the focal length to be F, the point (.theta.1, h1) of panoramic picture corresponding to point (u1, v1) in the synthesized picture is determined in the following formula. EQU .theta.1=arc tan(u1/F)+.theta.0 EQU h1=v1.multidot.cos(.theta.) [3]
Supposing the angle of field of view to be a, a desired synthesized picture is obtained by calculating about u1 corresponding from .theta.0-.varies./2 to .theta.0+.varies./2.
FIG. 12c shows an example of a picture. Reference numeral 1204 in FIG. 12b is a region in a panoramic picture corresponding to the cut-out picture of FIG. 12c.
In such picture synthesizing apparatus, however, since all pictures are present on the same plane and circular cylinder, processing using the information of undulations on the picture is impossible.
That is, in the picture synthesizing apparatus, when a background picture is handled, a sufficiently remote far-shot picture can be synthesized, but a picture mutually interfering with the foreground cannot be synthesized.
For example, as the foreground other than the background, when an object by computer graphics or the like is synthesized simultaneously, there is no undulation information in the background picture, it is impossible to process so as to synthesize to hide part of the foreground object by part of the background, or to vary the mode of collision depending on the shape of the background when the foreground object moves to collide against the background.
When handling the background of a short distance to the viewing point such as a room or a corner of street, mutual interference of background and foreground is effective to enhance the presence of the synthesizing effect.